9 Parametric vs non-parametric vs semiparametric

9.1 Parametric vs Non-parametric vs Semiparametric: Complete Comparison

9.2 Quick Reference Table

Aspect	Parametric	Non-parametric	Semiparametric
Definition	Assumes specific probability distribution/functional form	Makes no distributional assumptions	Combines parametric and non-parametric components
Assumptions	Strong assumptions about data distribution	Minimal assumptions (often just continuity)	Moderate assumptions (some parameters, some flexible)
Flexibility	Low (fixed functional form)	High (data-driven shape)	Medium (flexible in some aspects)
Interpretability	High (clear parameters)	Low to Medium	Medium (interpretable parameters + flexible parts)
Sample Size	Works well with small samples	Requires larger samples	Moderate sample size requirements
Computational Cost	Low	Medium to High	Medium

9.3 Detailed Comparison with R Examples

9.3.1 1. Parametric Methods

Definition: Methods that assume data follows a specific probability distribution or functional form with a finite number of parameters.

Key Characteristics: - Fixed functional form - Parameters have clear interpretations - Efficient when assumptions are met - Can fail badly when assumptions are violated

# Example 1: Linear Regression (Parametric)
# Assumes linear relationship between X and Y
set.seed(123)
x <- rnorm(100)
y <- 2 + 3*x + rnorm(100, 0, 0.5)  # True linear relationship

# Fit parametric model
lm_model <- lm(y ~ x)
summary(lm_model)

# Visualization
plot(x, y, main = "Parametric Linear Regression")
abline(lm_model, col = "red", lwd = 2)

# Example 2: Normal Distribution Test
# Assumes data follows normal distribution
data <- rnorm(1000, mean = 5, sd = 2)
shapiro.test(data)  # Parametric normality test

9.3.2 2. Non-parametric Methods

Definition: Methods that make minimal assumptions about the underlying data distribution and let the data determine the shape/relationship.

Key Characteristics: - No assumption about specific distribution - Flexible, adapts to data - Robust to outliers - Less efficient when parametric assumptions would be correct

# Example 1: Non-parametric Regression (LOESS)
# No assumption about functional form
set.seed(123)
x <- seq(0, 2*pi, length.out = 100)
y <- sin(x) + rnorm(100, 0, 0.1)  # Non-linear relationship

# Fit non-parametric model
loess_model <- loess(y ~ x)

# Visualization
plot(x, y, main = "Non-parametric LOESS Regression")
lines(x, predict(loess_model), col = "blue", lwd = 2, type = "l")

# Example 2: Non-parametric Density Estimation
data <- c(rnorm(500, 0, 1), rnorm(500, 3, 0.5))  # Bimodal data
dens <- density(data)  # Kernel density estimation
plot(dens, main = "Non-parametric Density Estimation")

# Example 3: Non-parametric Test
# Mann-Whitney U test (alternative to t-test)
group1 <- rnorm(50, 5, 2)
group2 <- rnorm(50, 7, 2)
wilcox.test(group1, group2)  # Non-parametric test

9.3.3 3. Semiparametric Methods

Definition: Methods that contain both parametric and non-parametric components, combining the interpretability of parametric models with the flexibility of non-parametric approaches.

Key Characteristics: - Some parameters are specified (interpretable) - Some components are estimated flexibly - Balance between interpretability and flexibility - Common in modern statistical modeling

# Example 1: Generalized Additive Model (GAM)
# Combines linear terms (parametric) with smooth terms (non-parametric)
library(mgcv)

set.seed(123)
n <- 300
x1 <- rnorm(n)
x2 <- seq(0, 4*pi, length.out = n)
y <- 2*x1 + sin(x2) + rnorm(n, 0, 0.2)

# Fit semiparametric model
gam_model <- gam(y ~ x1 + s(x2))  # x1 linear (parametric), x2 smooth (non-parametric)
summary(gam_model)

# Visualization
par(mfrow = c(1, 2))
plot(gam_model, select = 1, main = "Smooth term for x2")
plot(x1, y, main = "Linear term for x1")
abline(lm(y ~ x1), col = "red", lwd = 2)

# Example 2: Cox Proportional Hazards Model
# Parametric log-hazard, non-parametric baseline hazard
library(survival)

# Simulate survival data
set.seed(123)
n <- 200
x1 <- rbinom(n, 1, 0.5)  # Binary covariate
x2 <- rnorm(n)           # Continuous covariate
time <- rexp(n, exp(0.5*x1 + 0.3*x2))
event <- rbinom(n, 1, 0.8)

# Fit Cox model
cox_model <- coxph(Surv(time, event) ~ x1 + x2)
summary(cox_model)

9.4 When to Use Each Approach

9.4.1 Use Parametric When:

You have strong theoretical justification for the assumed form
Sample size is small
You need precise parameter estimates and their interpretations
Computational efficiency is important
You’re doing hypothesis testing with known distributions

# Example: t-test when you know data is approximately normal
group1 <- rnorm(30, 10, 2)
group2 <- rnorm(30, 12, 2)
t.test(group1, group2)  # Parametric test

9.4.2 Use Non-parametric When:

You don’t know the underlying distribution
Data violates parametric assumptions
You have outliers or non-standard data
You want robust results
Exploratory data analysis

# Example: Rank-based correlation when relationship isn't linear
x <- 1:50
y <- x^2 + rnorm(50, 0, 100)  # Quadratic relationship with noise
cor(x, y, method = "spearman")  # Non-parametric correlation

9.4.3 Use Semiparametric When:

You want some interpretable parameters but also flexibility
You have mixed types of predictors
You need to control for confounding non-parametrically
Doing survival analysis
Working with time series or longitudinal data

# Example: Mixed effects model with smooth terms
library(gamm4)
set.seed(123)
n <- 100
group <- rep(1:5, each = 20)
x <- runif(n, 0, 10)
y <- 2*sin(x/2) + rnorm(n, 0, 0.5) + rnorm(5, 0, 0.5)[group]

# Semiparametric mixed model
gamm_model <- gamm4(y ~ s(x), random = ~(1|group))

9.5 Summary

Parametric methods are like using a specific template - efficient and interpretable when the template fits your data, but restrictive when it doesn’t.

Non-parametric methods are like having complete freedom - flexible and robust, but may be overkill and harder to interpret.

Semiparametric methods are like having a customizable template - you get the best of both worlds with some structure and some flexibility.

Choose your approach based on your specific context, data characteristics, and analysis goals!